Multi-camera cooperative scene interpretation

In our society, video processing has become a convenient and widely used tool to assist, protect and simplify the daily life of people in areas such as surveillance and video conferencing. The growing number of cameras, the handling and analysis of these vast amounts of video data enable the development of multi-camera applications that cooperatively use multiple sensors. In many applications, bandwidth constraints, privacy issues, and difficulties in storing and analyzing large amounts of video data make applications costly and technically challenging. In this thesis, we deploy techniques ranging from low-level to high-level approaches, specifically designed for multi-camera networks. As a low-level approach, we designed a novel low-level foreground detection algorithm for real-time tracking applications, concentrating on difficult and changing illumination conditions. The main part of this dissertation focuses on a detailed analysis of two novel state-of-the-art real-time tracking approaches: a multi-camera tracking approach based on occupancy maps and a distributed multi-camera tracking approach with a feedback loop. As a high-level application we propose an approach to understand the dynamics in meetings - so called, smart meetings - using a multi-camera setup, consisting of fixed ambient and portable close-up cameras. For all method, we provided qualitative and quantitative results on several experiments, compared to state-of-the-art methods

A best view selection in meetings through attention analysis using a multi-camera network

Human activity analysis is an essential task in ambient intelligence and computer vision. The main focus lies in the automatic analysis of ongoing activities from a multi-camera network. One possible application is meeting analysis which explores the dynamics in meetings using low-level data and inferring high-level activities. However, the detection of such activities is still very challenging due to the often corrupted or imprecise low-level data. In this paper, we present an approach to understand the dynamics in meetings using a multi-camera network, consisting of fixed ambient and portable close-up cameras. As a particular application we are aiming to find the most informative video stream, for example as a representative view for a remote participant. Our contribution is threefold: at first, we estimate the extrinsic parameters of the portable close-up cameras based on head positions. Secondly, we find common overlapping areas based on the consensus of people’s orientation. And thirdly, the most informative view for a remote participant is estimated using common overlapping areas. We evaluated our proposed approach and compared it to a motion estimation method. Experimental results show that we can reach an accuracy of 74% compared to manually selected views

A canonical correlation analysis based motion model for probabilistic visual tracking

Particle filters are often used for tracking objects within a scene. As the prediction model of a particle filter is often implemented using basic movement predictions such as random walk, constant velocity or acceleration, these models will usually be incorrect. Therefore, this paper proposes a new approach, based on a Canonical Correlation Analysis (CCA) tracking method which provides an object specific motion model. This model is used to construct a proposal distribution of the prediction model which predicts new states, increasing the robustness of the particle filter. Results confirm an increase in accuracy compared to state-of-the-art method

Demo: real-time indoors people tracking in scalable camera networks

In this demo we present a people tracker in indoor environments. The tracker executes in a network of smart cameras with overlapping views. Special attention is given to real-time processing by distribution of tasks between the cameras and the fusion server. Each camera performs tasks of processing the images and tracking of people in the image plane. Instead of camera images, only metadata (a bounding box per person) are sent from each camera to the fusion server. The metadata are used on the server side to estimate the position of each person in real-world coordinates. Although the tracker is designed to suit any indoor environment, in this demo the tracker's performance is presented in a meeting scenario, where occlusions of people by other people and/or furniture are significant and occur frequently. Multiple cameras insure views from multiple angles, which keeps tracking accurate even in cases of severe occlusions in some of the views

PhD forum: multi-view occupancy maps using a network of low resolution visual sensors

An occupancy map provides an abstract top view of a scene and can be used for many applications such as domotics, surveillance, elderly-care and video teleconferencing. Such maps can be accurately estimated from multiple camera views. However, using a network of regular high resolution cameras makes the system expensive, and quickly raises privacy concerns (e. g. in elderly homes). Furthermore, their power consumption makes battery operation difficult. A solution could be the use of a network of low resolution visual sensors, but their limited resolution could degrade the accuracy of the maps. In this paper we used simulations to determine the minimum required resolution needed for deriving accurate occupancy maps which were then used to track people. Multi-view occupancy maps were computed from foreground silhouettes derived via an analysis of moving edges. Ground occupancies computed from each view were fused in a Dempster-Shafer framework. Tracking was done via a Bayes filter using the occupancy map per time instance as measurement. We found that for a room of 8.8 by 9.2 m, 4 cameras with a resolution as low as 64 by 48 pixels was sufficient to estimate accurate occupancy maps and track up to 4 people. These findings indicate that it is possible to use low resolution visual sensors to build a cheap, power efficient and privacy-friendly system for occupancy monitoring

PhD forum: correlation coefficient based template matching for indoor people tracking

Abstract—One of the most popular methods to extract information from an image sequence is template matching. The principle of template matching is tracking a certain feature or target over time based on the comparison of the content of each frame with a simple template. In this article, we propose an correlation coefficient based template matching which is invariant to linear intensity distortions to do correction or verification of our existing indoor people tracking system

PhD Forum: illumination-robust foreground detection for multi-camera occupancy mapping

Foreground detection is an essential preprocessing step for many image processing applications such as object tracking, human action recognition, pose estimation and occupancy mapping. Many existing techniques only perform well under steady illumination. Some approaches have been introduced to detect foreground under varying or sudden changes in illumination but the problem remains challenging. In this paper, we introduce a new texture-based foreground detection method which is robust to illumination change. Our method detects foreground by finding the correlation between the current frame and a background model. A region with low correlation is detected as foreground. We compare the performance of our proposed technique with other techniques from literature (edge-based, ViBe and Gaussian mixture model) as a preprocessing step of the multi-camera occupancy mapping system. The evaluation demonstrates that our technique outperforms the other methods in term of object loss

Robuste Lokalisierung von autonomen Fahrzeugen mittels Landmarken

Die Fahrzeuglokalisierung ist im Bereich der Fahrerassistenzsysteme von entscheidender Bedeutung und Voraussetzung f\u7fur verschiedene Anwendungen der Robotik, wie z.B. Navigation oder Kollisionsvermeidung f\u7fur fahrerlose Transportsysteme (FTS). In dieser Arbeit wird ein Verfahren zur Lokalisierung mittels Landmarken vorgestellt, die eine Orientierung bez\u7fuglich einer Karte erm\u7foglichen. Dabei werden der Erweiterte- Kalman-Filter und der Partikel-Filter f\u7fur diese Aufgabe untersucht und verglichen. Ein Schwerpunkt dieser Betrachtungen stellt dabei der Partikel-Filter dar. Die besondere Problematik der Initialisierung wird ausf\u7fuhrlich f\u7fur beide Filter dargestellt. Simulationen und Versuche zeigen, dass sich der Partikel-Filter f\u7fur eine robuste Lokalisierung der Fahrzeugposition verwenden l\u7fasst. Im Vergleich dazu kann der Erweiterte-Kalman-Filter nur im begrenzten Maße eingesetzt werden.The localization of vehicles is of vital importance in the field of driver assistance systems and a requirement of different applications for robotics, i.e. navigation or collision avoidance for automatic guided vehicle systems. In this thesis an approach for localization by means of landmarks is introduced, which enables an orientation regarding a map. The extended Kalman filter and the particle filter are analyzed and compared. The main focus for this consideration is on the particle filter. The problematic for initialization is discussed in detail for both filters. Simulations and tests prove that the particle filter is suitable for robust localization of the vehicle position. Compared to this, the extended Kalman filter can only be used to a certain extend

Efficient foreground detection for real-time surveillance applications

The problem of foreground detection in real-time video surveillance applications is addressed. Proposes is a framework, which is computationally cheap and has low memory requirements. It combines two simple processing blocks, both of which are essentially background subtraction algorithms. The main novelty of the approach is a combination of an autoregressive moving average filter with two background models having different adaptation speeds. The first model, having a lower adaptation speed, models long-term background and detects foreground objects by finding areas in the current frame which significantly differ from the proposed background model. The second model, with a higher adaptation speed, models the short-term background and is responsible for finding regions in the scene with a high foreground object activity. The final foreground detection is built by combining the outputs from these building blocks. The foreground obtained by the long-term modelling block is verified by the output of the short-term modelling block, i.e. only the objects exhibiting significant motion are detected as real foreground objects. The proposed method results in a very good foreground detection performance at a low computational cost